For example, Patent Document 1 (Japanese Patent Laid-open No. 10-222694 [G06T 15/00, 15/40]) recites that information on a degree of influence which image data of each pixel imparts on the surrounding pixels is stored as a blur value in a unit of pixel, and blur processing according to the blur value for each pixel is performed, and whereby an image on which blur processing is performed is displayed. The blur value is evaluated by computing a difference, etc. between a Z value for each pixel and depth of field.
Furthermore, Patent Document 2 (Japanese Patent Laid-open No. 11-242753 [G06T 15/00, 17/00, 15/40]) recites that on the basis of a deviation between a Z value held in each pixel of the three-dimensional graphics and a depth of field, a copy pixel distance of the pixel is calculated, and a luminance value of the pixel is equally divided and distributed to a plurality of adjacent pixels away from the pixel by the pixel copying distance in a predetermined direction, and whereby, a blur processing is performed.
The above-described Patent Document 1 is for calculating a degree of influence of each pixel imparted on the surrounding pixels as a blur value, but is not for calculating a range where each pixel is blurred. That is, in the Patent Document 1, the blurring range where each pixel is blurred is fixedly decided to be the surrounding pixels. Thus, the blurring ranges of respective pixels become uniform, resulting in an unnatural blur expression.
Furthermore, the above-described Patent Document 2 is for deciding adjacent pixels in which luminance is dispersed depending on the pixel copying distance calculated on the basis of a Z value of each pixel and the depth of field deviation, and is not for calculating a blurring range. That is, in the Patent Document 2, the number of adjacent pixels in which luminance is dispersed is decided in advance (four, for example). Accordingly, the Patent Document 2 also has a problem the same as the Patent Document 1.
Therefore, it is a primary feature of the present embodiment to provide a novel image processing program and image processing apparatus.
Another feature of the present embodiment is to provide an image processing program and an image processing apparatus capable of realizing a natural blur expression.
The present embodiment employs following features in order to solve the above-described problems. It should be noted that reference numerals inside the parentheses and supplements show one example of a corresponding relationship with the embodiments described later for easy understanding of the present embodiment, and do not limit the present embodiment.
A storage medium which machine-readably stores an image processing program by a computer of an image processing apparatus for displaying a virtual three-dimensional image by utilizing rendering data including color data and a depth value for each pixel, the image processing program causes the computer to execute a detecting step for detecting a focused position, a blur range calculating step for calculating for each pixel a difference between a depth value of the pixel and a depth value of the focused position which are included in the rendering data, and calculating a blurring range on the basis of the difference, a color data generating step for generating color data which is subjected to blurring processing as to blurring objective pixels specified on the basis of the blurring range, and a display image generating step for generating display image data on the basis of the color data generated by the color data generating step.
In the first embodiment, an image processing program is executed by a computer (40) of a game apparatus (12) in this embodiment. The computer (40) reads rendering data from a rendering buffer (136), and displays a virtual three-dimensional image by utilizing color data (R0, G0, B0) and a depth value (Z) for each pixel which are included in the rendering data. In a detecting step (S23), the computer detects a focused position when a virtual three-dimensional image is photographed on the basis of the operation information, for example. In a blur range calculating step (S25, S29), the computer calculates, for each pixel, a difference between a depth value (DZ) of the pixel (Dn) included in the rendering data and a depth value (PZ) of the focused position read from the rendering data, and calculates a blurring range on the basis of the foregoing difference. In a color data generating step (S31, S33, S35, S37), as to a blurring objective pixel included in the blurring range, by a method of modifying the color data included in the rendering data by a diffusion value, for example, color data on which blur processing is performed is generated. Here, the diffusion value relates to a value of a degree of blur for each pixel, and is a value relating to the foregoing difference, for example. The generated color data is written to the calculation buffer, for example, for each pixel. In a display image generating step (S43), the computer generates display image data (output color data) including a blurred image on the basis of the color data of each of the pixels stored in the rendering buffer.
According to the first embodiment, the blurring objective pixel is specified by the blurring range calculated on the basis of the difference between the depth value (DZ) of each pixel (Dn) and the depth value (PZ) of the focused position, and color data which is subjected to the blur processing depending on the difference for each blurring objective pixel is generated, and therefore, it is possible to realize a more natural blur expression.
A second embodiment is a storage medium, wherein the color data generating step of the image processing program includes a blurring objective pixel specifying step for specifying a blurring objective pixel.
In the second embodiment, the computer executes a blurring objective pixel specifying step (S35, S37) in the color data generating step to specify an objective pixel on which blur processing has to be performed. According to the second embodiment, it is easily specify the pixels as a blur object.
A third embodiment is a storage medium, wherein the blurring objective pixel specifying step of the image processing program includes a matching step for matching the blurring range and blurring shape data set in advance to specify the blurring objective pixels.
In the third embodiment, the computer specifies blurring objective pixels by executing a matching step (S35) for matching a blurring range and the blurring shape data (hexagon, for example) set in advance. According to the third embodiment, by changing the blurring shape, it is possible to easily change the range (or the shape of the range) of the blurring objective pixels.
A fourth embodiment is a storage medium, and wherein the color data generating step of the image processing program includes a diffusion value calculating step for calculating a diffusion value of each of the pixels included in the blurring range calculated by the blur range calculating step and a color data changing step for changing, on the basis of the diffusion value, the color data included in the rendering data of the pixel determined as a blur object by the matching step.
In the fourth embodiment, in a diffusion value calculating step (S31, S33), the computer calculates a diffusion value (DR, DG, DB) of each of the pixels included in the blurring range calculated by the blur range calculating step. In a color data changing step (S37), the color data included in the rendering data of the pixel determined as a blur object by the matching step (S35) is changed on the basis of the diffusion value. For example, R1=R1+DR, G1=G1+DG, and B1=B1+DB. The weight data is H+DH. According to the fourth invention, it is possible to easily calculate color data (and weight data as required) on which blur processing is performed depending on the degree of the blur.
A fifth embodiment is a storage medium, wherein the diffusion value calculating step of the image processing program includes a weight calculating step for calculating a weight on the basis of an arithmetic expression which is proportion to a size of the color data of each pixel included in the rendering data and is inversely proportion to a size of the blurring range, and the diffusion value is calculated on the basis of the weight.
In the fifth embodiment, the computer first calculates a weight (DH), and calculates a diffusion value on the basis of the weight. In this embodiment, each of the color components R, G, and B is regarded as a vector, and the strength of the light is represented by the magnitude. The coefficient value representing a magnitude of the so-called “color vector” is “weight”. According to the fifth embodiment, since the degree of blurring the pixels can be calculated so as to be in proportion to the size of the color data and is inversely proportion to the size of the blurring range, it is possible to make the blur expression more natural.
A sixth embodiment is a storage medium, wherein the image processing program causes the computer to further execute a determining step for determining whether or not a photographing instruction is input, the detecting step detects a focused position when it is determined by the determining step that a photographing instruction is input, and the color data generating step includes a color data changing step for changing the color data included in the rendering data on the basis of a diffusion value with respect to the blurring objective pixels.
In the sixth embodiment, a photographing instruction can be input to the computer by operating an operating device (the controller for the game apparatus) by the user, for example, and the computer determines whether or not such a photographing instruction is input by a determining step (S9). When determining that a photographing instruction is input, the computer detects a focused position when a virtual three-dimensional image is photographed on the basis of the operation information in a detecting step (S23). A color data generating step changes color data included in the rendering data for each of the blurring objective pixels on the basis of the diffusion value to thereby generate color data which is subjected to the blur processing. According to the sixth embodiment, it is possible to easily evaluate color data for a blurred image.
A seventh embodiment is a storage medium, and wherein the image processing program causes the computer to further execute an operation information acquiring step for acquiring operation information by the user, wherein the detecting step specifies a pixel designated on the basis of the operation information, and detects a depth value of the pixel as the focused position.
In the seventh embodiment, the computer fetches operation information by the user in an operation information acquiring step (S5), and specifies a pixel which the user designates as a pixel to be focused on the basis of the operation information, and detects a depth value of the pixel as a focused position in a detecting step (S23). According to the seventh embodiment, it is possible to make the user freely select an arbitrary position as a focused position in the virtual three-dimensional image.
An eighth embodiment is an image processing apparatus which displays a virtual three-dimensional image by utilizing rendering data including color data and a depth value for each pixel, comprises: a detecting means for detecting a focused position, a blurring range calculating means for calculating for each pixel a difference between a depth value of the pixel and a depth value of the focused position which are included in the rendering data, and calculating a blurring range on the basis of the difference, a color data generating means for generating color data which is subjected to blur processing with respect to blurring objective pixels specified on the basis of the blurring range, and a display image generating means for generating display image data on the basis of the color data generated by the color data generating means.
In the eighth embodiment, a game apparatus (12) in this embodiment is assumed as an image processing apparatus. The computer (40) of the game apparatus reads rendering data from a rendering buffer (136), for example, and displays a virtual three-dimensional image (game image) by utilizing color data (R0, G0, B0) for each pixel and a depth value (Z) which are included in the rendering data. Furthermore, each of the means can be specified as a program step to be executed by the computer. For example, in response to a photographing instruction, the detecting means (40, S23) detects a focused position when a virtual three-dimensional image is photographed. A blurring range calculation means (S25, S29) calculates, for each pixel, a difference between a depth value (DZ) of the pixel (Dn) included in the rendering data and a depth value (PZ) of the focused position read from the rendering data, and calculates a blurring range on the basis of the above-described difference. By a color data generating means (S31, S33, S35, S37), as to blurring objective pixels included in the blurring range, by a method of modifying the color data included in the rendering data by a diffusion value, for example, color data on which blur processing is performed is generated, and the generated color data is written to the calculation buffer for each pixel, for example. A display image generating means (S43) generates display image data (output color data) including a blurred image on the basis of the color data of each pixel stored in the rendering buffer.
The eighth embodiment has an advantage similar to the above-described image processing program of the first embodiment.
According to the present embodiment, since color data which is subjected to the blur processing as to the blurring objective pixel specified by the blurring range calculated between a depth value of each pixel and a depth value of the focused position, it is possible to realize a more natural blur expression.
The above described features, aspects and advantages of the present embodiment will become more apparent from the following detailed description of the present embodiment when taken in conjunction with the accompanying drawings.